1. Technical Field
The present invention relates in general to methods and system for enhancing the usability of computer systems by blind or visually impaired users, and more particularly to methods and systems for permitting blind or visually impaired users to use a graphical user interface. Still more particularly, the present invention relates to methods and systems for enabling a blind or visually impaired user to graphically select displayed elements in a graphical user interface.
2. Description of the Related Art
In recent years, there has been a move among computer application software developers toward graphical user interfaces. In a graphical user interface, objects are presented for users to manipulate in ways which are similar to the manner in which such objects are manipulated in the work place. Objects, such as file cabinets, folders, documents, and printers, are displayed on the computer screen as miniature graphic representations, or icons. Users then manipulate these objects with a mouse or other graphical pointing device to perform desired operations.
For example, in order to file a document in a folder which is located within a file cabinet in the work place, a user will open the file cabinet, locate and open the correct folder, and then place the document inside. In the electronic work place of the graphical user interface, the user performs a similar process. Thus, a user will open the file cabinet icon, locate the correct folder icon, and then drop the document icon in the located folder. Because this is an electronic environment, users do not have to open the folder to put the document into it. However, users are thus able to utilize their knowledge of a real work place in order to perform this operation.
Normally sighted individuals find graphical user interfaces intuitive and easy to work with. However, except for an occasional "beep" or similar tone, graphical user interfaces are virtually silent and the vast majority of the information which such interfaces provide to the user is visual. Thus, graphical user interfaces are essentially not usable by blind or severely visually impaired people.
Blind and visually impaired computer users currently benefit from many forms of adaptive technology, including speech synthesis, large-print processing, braille desk top publishing, and voice recognition. However, presently almost none of the foregoing tools have been adapted for use with a graphical user interface. It has been suggested that programmers could write software with built-in voice labels for icons. Lazzaro, Windows of Vulnerability, Byte Magazine, June 1991, page 416.
Various synthetic or recorded speech solutions for making computer display contents available to blind persons have also been suggested, for example in Golding et al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10B, pages 5633-5636 (March 1984); and, Barnett et al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10A, pages 4950-4951 (March 1984).
Additionally, there have been suggested systems which include a mouse with a braille transducer so that a blind mouse user may read text and obtain certain tactile position feedback from such a mouse. Comerford, IBM Technical Disclosure Bulletin No. 3, Vol. 28, page 1343 (August 1985); and Affinito, et al., IBM Technical Disclosure Bulletin No. 12, Vol. 31, page 386 (May 1989). However, while announcing various text items, either audibly or by means of a braille transducer in the mouse, such systems may provide some information to a blind user, they do not enable the user to navigate about and locate objects on the computer display screen.
There have been suggested an audible cursor positioning and pixel (picture element) status identification mechanism which may be utilized to help a user of an interactive computer graphics system locate data by utilizing aural feedback to enhance visual feedback. As the cursor in such a system is stepped across the screen, an audible click is generated which varies in tone corresponding in tone to the current status of each pixel encountered. With this combination in audible and visual cursor feedback, it becomes a simple task to identify the desired line by noting the change in tone as the cursor moves. For color display applications, each color is represented by a distinct tone so that any single pixel may be distinguished from surrounding pixels of a different color. It has been suggested that this system is especially helpful for visually impaired or learning disabled users. Drumm et al., IBM Technical Disclosure Bulletin No. 48, Vol. 27, page 25-28 (September 1984). However, the foregoing disclosure does not suggest a means of enabling a blind user to navigate about or locate objects on a computer display screen.
Recently, in one of the cross-referenced applications cited herein, a system has been proposed which permits a blind or visually impaired user to interact with a so-called "message box" within a graphical user interface. As those skilled in the art will appreciate, each message box consists of an icon, explanatory text, and one or more "push buttons." The icon allows the user to identify visually the type of message. The text typically explains the situation and may provide assistance. The textual content may be a question or a statement. Push buttons provided within a message box typically allow the user to interact with the message box.
The proposed system permits blind or visually impaired users to accommodate a message box by announcing the textual contents of such a box when the message box first appears. Thereafter, the push buttons available to respond to the message box are also announced in order from left to right. A homing signal is then provided for finding the message box which increases in pitch as the mouse pointer approaches the message box. When the pointer enters the message box, the message box text and available push buttons are reannounced and the pointer is automatically moved to a default push button. By utilizing this system, a blind or visually impaired user may locate a message box within a computer display system; however, this system fails to provide any suggestion of a manner in which a blind or visually impaired user may selectively locate multiple displayed elements within a graphical user interface.
Another method and system have also been recently proposed in the second cross-referenced application cited herein, which permits a blind or visually impaired user to locate a mouse pointer or other graphical pointing device within the client area of a window within a graphical user interface by providing a stereo sound system and varying the intensity of the left and right audio channels to indicate a position of the mouse pointer. This system also proposes an increase in pitch of an associated sound to indicate the position of the pointer in the top or bottom of the client area of a window. While this system permits a blind or visually impaired user to manipulate a mouse pointer within a graphical user interface, it fails to show or suggest any technique whereby a particular one of a group of displayed graphical elements may be selected by such a user.
Thus, it should be apparent that a need exists for a method and system whereby a blind or visually impaired user may graphically select displayed elements within a graphical user interface.